When video is streamed over the Internet and played back through a Web browser or media player, the video is delivered in digital form. Digital video is also used when video is delivered through many broadcast services, satellite services and cable television services. Real-time videoconferencing often uses digital video, and digital video is used during video capture with most smartphones, Web cameras and other video capture devices.
Digital video can consume an extremely high amount of bits. The number of bits that is used per second of represented video content is known as the bit rate. Engineers use compression (also called source coding or source encoding) to reduce the bit rate of digital video. Compression decreases the cost of storing and transmitting video information by converting the information into a lower bit rate form. Decompression (also called decoding) reconstructs a version of the original information from the compressed form. A “codec” is an encoder/decoder system.
Over the last two decades, various video codec standards have been adopted, including the H.261, H.262 (MPEG-2 or ISO/IEC 13818-2), H.263, H.264 (AVC or ISO/IEC 14496-10), and H.265 (HEVC) standards, the MPEG-1 (ISO/IEC 11172-2) and MPEG-4 Visual (ISO/IEC 14496-2) standards, and the SMPTE 421M (VC-1) standard. Various other formats have also been adopted, including VPx formats (e.g., VP6, VP8, VP9). In particular, decoding according to the VP8 format and decoding according to the VP9 format are often used to play back encoded video that is offered over the World Wide Web. VP8 and VP9 decoding are also used in some set-top boxes, personal computers, smartphones and other mobile computing devices for playback of encoded video streamed over the Internet or other networks. A video codec standard or format typically defines options for the syntax of an encoded video bitstream, detailing parameters in the bitstream when particular features are used in encoding and decoding. In many cases, a video codec standard or format also provides details about the decoding operations a decoder should perform to achieve correct results in decoding. In some cases, reference software provides an example of correct encoding and/or decoding according to a video codec standard or format.
While some video decoding operations are relatively simple, others are computationally complex. For example, inverse frequency transforms, fractional sample interpolation operations for motion compensation, in-loop deblock filtering, post-processing filtering, color conversion, and video re-sizing can require extensive computation. This computational complexity can be problematic in various scenarios, such as decoding of high-quality, high-bit rate video (e.g., compressed high-definition video).
Thus, some decoders use hardware acceleration to offload certain computationally intensive operations to a graphics processor or other special-purpose hardware. For example, in some configurations, a computer system includes a primary central processing unit (“CPU”) as well as a graphics processing unit (“GPU”) or other hardware specially adapted for graphics processing or video decoding. A decoder uses the primary CPU as a host decoder to control overall decoding and uses the GPU (or special-purpose decoding hardware) to perform operations that collectively require extensive computation, accomplishing video acceleration. In a typical software architecture for hardware-accelerated video decoding, a host decoder controls overall decoding and may perform some operations such as bitstream parsing using the CPU. The decoder signals control information (e.g., picture parameters, slice parameters) and encoded data to a device driver for an accelerator (e.g., with GPU) across an acceleration interface. Some existing hardware acceleration architectures are adapted for decoding according to standards such as H.264, but they do not address features of decoding according to certain formats such as VP8 and VP9.